{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8e8743c8",
   "metadata": {},
   "source": [
    "# Using PEFT with custom models"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c42c67e1",
   "metadata": {},
   "source": [
    "`peft` allows us to fine-tune models efficiently with LoRA. In this short notebook, we will demonstrate how to train a simple multilayer perceptron (MLP) using `peft`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce314af5",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b28b214d",
   "metadata": {},
   "source": [
    "Make sure that you have the latest version of `peft` installed. To ensure that, run this in your Python environment:\n",
    "    \n",
    "    python -m pip install --upgrade peft"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "4d9da3d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "import copy\n",
    "import os\n",
    "\n",
    "# ignore bnb warnings\n",
    "os.environ[\"BITSANDBYTES_NOWELCOME\"] = \"1\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "44075f54",
   "metadata": {},
   "outputs": [],
   "source": [
    "import peft\n",
    "import torch\n",
    "from torch import nn\n",
    "import torch.nn.functional as F"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "f72acdfb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x7f2a64177510>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.manual_seed(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b127a78",
   "metadata": {},
   "source": [
    "## Data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f265da76",
   "metadata": {},
   "source": [
    "We will create a toy dataset consisting of random data for a classification task. There is a little bit of signal in the data, so we should expect that the loss of the model can improve during training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b355567e",
   "metadata": {},
   "outputs": [],
   "source": [
    "X = torch.rand((1000, 20))\n",
    "y = (X.sum(1) > 10).long()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a60a869d",
   "metadata": {},
   "outputs": [],
   "source": [
    "n_train = 800\n",
    "batch_size = 64"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "8859572e",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_dataloader = torch.utils.data.DataLoader(\n",
    "    torch.utils.data.TensorDataset(X[:n_train], y[:n_train]),\n",
    "    batch_size=batch_size,\n",
    "    shuffle=True,\n",
    ")\n",
    "eval_dataloader = torch.utils.data.DataLoader(\n",
    "    torch.utils.data.TensorDataset(X[n_train:], y[n_train:]),\n",
    "    batch_size=batch_size,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97bddd2c",
   "metadata": {},
   "source": [
    "## Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db694a58",
   "metadata": {},
   "source": [
    "As a model, we use a simple multilayer perceptron (MLP). For demonstration purposes, we use a very large number of hidden units. This is totally overkill for this task but it helps to demonstrate the advantages of `peft`. In more realistic settings, models will also be quite large on average, so this is not far-fetched."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "1b43cd8f",
   "metadata": {},
   "outputs": [],
   "source": [
    "class MLP(nn.Module):\n",
    "    def __init__(self, num_units_hidden=2000):\n",
    "        super().__init__()\n",
    "        self.seq = nn.Sequential(\n",
    "            nn.Linear(20, num_units_hidden),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(num_units_hidden, num_units_hidden),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(num_units_hidden, 2),\n",
    "            nn.LogSoftmax(dim=-1),\n",
    "        )\n",
    "\n",
    "    def forward(self, X):\n",
    "        return self.seq(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1277bf00",
   "metadata": {},
   "source": [
    "## Training"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02caf26a",
   "metadata": {},
   "source": [
    "Here are just a few training hyper-parameters and a simple function that performs the training and evaluation loop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "5d14c0c4",
   "metadata": {},
   "outputs": [],
   "source": [
    "lr = 0.002\n",
    "batch_size = 64\n",
    "max_epochs = 30\n",
    "device = \"cpu\" if not torch.cuda.is_available() else \"cuda\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "657d6b3e",
   "metadata": {},
   "outputs": [],
   "source": [
    "def train(model, optimizer, criterion, train_dataloader, eval_dataloader, epochs):\n",
    "    for epoch in range(epochs):\n",
    "        model.train()\n",
    "        train_loss = 0\n",
    "        for xb, yb in train_dataloader:\n",
    "            xb = xb.to(device)\n",
    "            yb = yb.to(device)\n",
    "            outputs = model(xb)\n",
    "            loss = criterion(outputs, yb)\n",
    "            train_loss += loss.detach().float()\n",
    "            loss.backward()\n",
    "            optimizer.step()\n",
    "            optimizer.zero_grad()\n",
    "\n",
    "        model.eval()\n",
    "        eval_loss = 0\n",
    "        for xb, yb in eval_dataloader:\n",
    "            xb = xb.to(device)\n",
    "            yb = yb.to(device)\n",
    "            with torch.no_grad():\n",
    "                outputs = model(xb)\n",
    "            loss = criterion(outputs, yb)\n",
    "            eval_loss += loss.detach().float()\n",
    "\n",
    "        eval_loss_total = (eval_loss / len(eval_dataloader)).item()\n",
    "        train_loss_total = (train_loss / len(train_dataloader)).item()\n",
    "        print(f\"{epoch=:<2}  {train_loss_total=:.4f}  {eval_loss_total=:.4f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b382dcbe",
   "metadata": {},
   "source": [
    "### Training without peft"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b40d4873",
   "metadata": {},
   "source": [
    "Let's start without using `peft` to see what we can expect from the model training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "f059ced4",
   "metadata": {},
   "outputs": [],
   "source": [
    "module = MLP().to(device)\n",
    "optimizer = torch.optim.Adam(module.parameters(), lr=lr)\n",
    "criterion = nn.CrossEntropyLoss()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "17698863",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch=0   train_loss_total=0.7970  eval_loss_total=0.6472\n",
      "epoch=1   train_loss_total=0.5597  eval_loss_total=0.4898\n",
      "epoch=2   train_loss_total=0.3696  eval_loss_total=0.3323\n",
      "epoch=3   train_loss_total=0.2364  eval_loss_total=0.5454\n",
      "epoch=4   train_loss_total=0.2428  eval_loss_total=0.2843\n",
      "epoch=5   train_loss_total=0.1251  eval_loss_total=0.2514\n",
      "epoch=6   train_loss_total=0.0952  eval_loss_total=0.2068\n",
      "epoch=7   train_loss_total=0.0831  eval_loss_total=0.2395\n",
      "epoch=8   train_loss_total=0.0655  eval_loss_total=0.2524\n",
      "epoch=9   train_loss_total=0.0380  eval_loss_total=0.3650\n",
      "epoch=10  train_loss_total=0.0363  eval_loss_total=0.3495\n",
      "epoch=11  train_loss_total=0.0231  eval_loss_total=0.2360\n",
      "epoch=12  train_loss_total=0.0162  eval_loss_total=0.2276\n",
      "epoch=13  train_loss_total=0.0094  eval_loss_total=0.2716\n",
      "epoch=14  train_loss_total=0.0065  eval_loss_total=0.2237\n",
      "epoch=15  train_loss_total=0.0054  eval_loss_total=0.2366\n",
      "epoch=16  train_loss_total=0.0035  eval_loss_total=0.2673\n",
      "epoch=17  train_loss_total=0.0028  eval_loss_total=0.2630\n",
      "epoch=18  train_loss_total=0.0023  eval_loss_total=0.2835\n",
      "epoch=19  train_loss_total=0.0021  eval_loss_total=0.2727\n",
      "epoch=20  train_loss_total=0.0018  eval_loss_total=0.2597\n",
      "epoch=21  train_loss_total=0.0016  eval_loss_total=0.2553\n",
      "epoch=22  train_loss_total=0.0014  eval_loss_total=0.2712\n",
      "epoch=23  train_loss_total=0.0013  eval_loss_total=0.2637\n",
      "epoch=24  train_loss_total=0.0012  eval_loss_total=0.2733\n",
      "epoch=25  train_loss_total=0.0011  eval_loss_total=0.2738\n",
      "epoch=26  train_loss_total=0.0010  eval_loss_total=0.2476\n",
      "epoch=27  train_loss_total=0.0010  eval_loss_total=0.2583\n",
      "epoch=28  train_loss_total=0.0009  eval_loss_total=0.2842\n",
      "epoch=29  train_loss_total=0.0008  eval_loss_total=0.2634\n",
      "CPU times: user 1.26 s, sys: 187 ms, total: 1.45 s\n",
      "Wall time: 1.45 s\n"
     ]
    }
   ],
   "source": [
    "%time train(module, optimizer, criterion, train_dataloader, eval_dataloader, epochs=max_epochs)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4cef0029",
   "metadata": {},
   "source": [
    "Okay, so we got an eval loss of ~0.26, which is much better than random."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f106078",
   "metadata": {},
   "source": [
    "### Training with peft"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8dd47aa4",
   "metadata": {},
   "source": [
    "Now let's train with `peft`. First we check the names of the modules, so that we can configure `peft` to fine-tune the right modules."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "922db29b",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('', __main__.MLP),\n",
       " ('seq', torch.nn.modules.container.Sequential),\n",
       " ('seq.0', torch.nn.modules.linear.Linear),\n",
       " ('seq.1', torch.nn.modules.activation.ReLU),\n",
       " ('seq.2', torch.nn.modules.linear.Linear),\n",
       " ('seq.3', torch.nn.modules.activation.ReLU),\n",
       " ('seq.4', torch.nn.modules.linear.Linear),\n",
       " ('seq.5', torch.nn.modules.activation.LogSoftmax)]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[(n, type(m)) for n, m in MLP().named_modules()]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5efb275d",
   "metadata": {},
   "source": [
    "Next we can define the LoRA config. There is nothing special going on here. We set the LoRA rank to 8 and select the layers `seq.0` and `seq.2` to be used for LoRA fine-tuning. As for `seq.4`, which is the output layer, we set it as `module_to_save`, which means it is also trained but no LoRA is applied."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf2c608d",
   "metadata": {},
   "source": [
    "*Note: Not all layers types can be fine-tuned with LoRA. At the moment, linear layers, embeddings, `Conv2D` and `transformers.pytorch_utils.Conv1D` are supported."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "b342438f",
   "metadata": {},
   "outputs": [],
   "source": [
    "config = peft.LoraConfig(\n",
    "    r=8,\n",
    "    target_modules=[\"seq.0\", \"seq.2\"],\n",
    "    modules_to_save=[\"seq.4\"],\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "829b4e2d",
   "metadata": {},
   "source": [
    "Now let's create the `peft` model by passing our initial MLP, as well as the config we just defined, to `get_peft_model`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "602b6658",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "trainable params: 56,164 || all params: 4,100,164 || trainable%: 1.369798866581922\n"
     ]
    }
   ],
   "source": [
    "module = MLP().to(device)\n",
    "module_copy = copy.deepcopy(module)  # we keep a copy of the original model for later\n",
    "peft_model = peft.get_peft_model(module, config)\n",
    "optimizer = torch.optim.Adam(peft_model.parameters(), lr=lr)\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "peft_model.print_trainable_parameters()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2103737d",
   "metadata": {},
   "source": [
    "Checking the numbers, we see that only ~1% of parameters are actually trained, which is what we like to see.\n",
    "\n",
    "Now let's start the training:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "9200cbc6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch=0   train_loss_total=0.6918  eval_loss_total=0.6518\n",
      "epoch=1   train_loss_total=0.5975  eval_loss_total=0.6125\n",
      "epoch=2   train_loss_total=0.5402  eval_loss_total=0.4929\n",
      "epoch=3   train_loss_total=0.3886  eval_loss_total=0.3476\n",
      "epoch=4   train_loss_total=0.2677  eval_loss_total=0.3185\n",
      "epoch=5   train_loss_total=0.1938  eval_loss_total=0.2294\n",
      "epoch=6   train_loss_total=0.1712  eval_loss_total=0.2653\n",
      "epoch=7   train_loss_total=0.1555  eval_loss_total=0.2764\n",
      "epoch=8   train_loss_total=0.1218  eval_loss_total=0.2104\n",
      "epoch=9   train_loss_total=0.0846  eval_loss_total=0.1756\n",
      "epoch=10  train_loss_total=0.0710  eval_loss_total=0.1873\n",
      "epoch=11  train_loss_total=0.0372  eval_loss_total=0.1539\n",
      "epoch=12  train_loss_total=0.0350  eval_loss_total=0.2348\n",
      "epoch=13  train_loss_total=0.0298  eval_loss_total=0.4605\n",
      "epoch=14  train_loss_total=0.0355  eval_loss_total=0.2208\n",
      "epoch=15  train_loss_total=0.0099  eval_loss_total=0.1583\n",
      "epoch=16  train_loss_total=0.0051  eval_loss_total=0.2042\n",
      "epoch=17  train_loss_total=0.0029  eval_loss_total=0.2045\n",
      "epoch=18  train_loss_total=0.0022  eval_loss_total=0.2285\n",
      "epoch=19  train_loss_total=0.0015  eval_loss_total=0.2118\n",
      "epoch=20  train_loss_total=0.0012  eval_loss_total=0.2237\n",
      "epoch=21  train_loss_total=0.0010  eval_loss_total=0.2363\n",
      "epoch=22  train_loss_total=0.0009  eval_loss_total=0.2531\n",
      "epoch=23  train_loss_total=0.0008  eval_loss_total=0.2528\n",
      "epoch=24  train_loss_total=0.0007  eval_loss_total=0.2443\n",
      "epoch=25  train_loss_total=0.0006  eval_loss_total=0.2267\n",
      "epoch=26  train_loss_total=0.0006  eval_loss_total=0.2379\n",
      "epoch=27  train_loss_total=0.0005  eval_loss_total=0.2658\n",
      "epoch=28  train_loss_total=0.0005  eval_loss_total=0.2326\n",
      "epoch=29  train_loss_total=0.0004  eval_loss_total=0.2520\n",
      "CPU times: user 950 ms, sys: 4.7 ms, total: 955 ms\n",
      "Wall time: 957 ms\n"
     ]
    }
   ],
   "source": [
    "%time train(peft_model, optimizer, criterion, train_dataloader, eval_dataloader, epochs=max_epochs)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20f6f452",
   "metadata": {},
   "source": [
    "In the end, we see that the eval loss is very similar to the one we saw earlier when we trained without `peft`. This is quite nice to see, given that we are training a much smaller number of parameters."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa55d1d4",
   "metadata": {},
   "source": [
    "#### Check which parameters were updated"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6e2146b",
   "metadata": {},
   "source": [
    "Finally, just to check that LoRA was applied as expected, we check what original weights were updated what weights stayed the same."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "c7dcde21",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New parameter model.seq.0.lora_A.default.weight |   160 parameters | updated\n",
      "New parameter model.seq.0.lora_B.default.weight | 16000 parameters | updated\n",
      "New parameter model.seq.2.lora_A.default.weight | 16000 parameters | updated\n",
      "New parameter model.seq.2.lora_B.default.weight | 16000 parameters | updated\n"
     ]
    }
   ],
   "source": [
    "for name, param in peft_model.base_model.named_parameters():\n",
    "    if \"lora\" not in name:\n",
    "        continue\n",
    "\n",
    "    print(f\"New parameter {name:<13} | {param.numel():>5} parameters | updated\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "022e6c41",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Parameter seq.0.weight  |   40000 parameters | not updated\n",
      "Parameter seq.0.bias    |    2000 parameters | not updated\n",
      "Parameter seq.2.weight  | 4000000 parameters | not updated\n",
      "Parameter seq.2.bias    |    2000 parameters | not updated\n",
      "Parameter seq.4.weight  |    4000 parameters | not updated\n",
      "Parameter seq.4.bias    |       2 parameters | not updated\n",
      "Parameter seq.4.weight  |    4000 parameters | updated\n",
      "Parameter seq.4.bias    |       2 parameters | updated\n"
     ]
    }
   ],
   "source": [
    "params_before = dict(module_copy.named_parameters())\n",
    "for name, param in peft_model.base_model.named_parameters():\n",
    "    if \"lora\" in name:\n",
    "        continue\n",
    "\n",
    "    name_before = (\n",
    "        name.partition(\".\")[-1].replace(\"original_\", \"\").replace(\"module.\", \"\").replace(\"modules_to_save.default.\", \"\")\n",
    "    )\n",
    "    param_before = params_before[name_before]\n",
    "    if torch.allclose(param, param_before):\n",
    "        print(f\"Parameter {name_before:<13} | {param.numel():>7} parameters | not updated\")\n",
    "    else:\n",
    "        print(f\"Parameter {name_before:<13} | {param.numel():>7} parameters | updated\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c09b43d",
   "metadata": {},
   "source": [
    "So we can see that apart from the new LoRA weights that were added, only the last layer was updated. Since the LoRA weights and the last layer have comparitively few parameters, this gives us a big boost in efficiency."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b46c6198",
   "metadata": {},
   "source": [
    "## Sharing the model through Hugging Face Hub"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6289e647",
   "metadata": {},
   "source": [
    "### Pushing the model to HF Hub"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06dcdfa0",
   "metadata": {},
   "source": [
    "With the `peft` model, it is also very easy to push a model the Hugging Face Hub. Below, we demonstrate how it works. It is assumed that you have a valid Hugging Face account and are logged in:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "1b91a0af",
   "metadata": {},
   "outputs": [],
   "source": [
    "user = \"BenjaminB\"  # put your user name here\n",
    "model_name = \"peft-lora-with-custom-model\"\n",
    "model_id = f\"{user}/{model_name}\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "1430fffd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8163dba7aa8e4012830d72fd7342e9b6",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "adapter_model.bin:   0%|          | 0.00/211k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5370fdac247e4a4180406a59e5f1ed63",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Upload 1 LFS files:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "peft_model.push_to_hub(model_id);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "632bd799",
   "metadata": {},
   "source": [
    "As we can see, the adapter size is only 211 kB."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ff78c0c",
   "metadata": {},
   "source": [
    "### Loading the model from HF Hub"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5c7e87f",
   "metadata": {},
   "source": [
    "Now, it only takes one step to load the model from HF Hub. To do this, we can use `PeftModel.from_pretrained`, passing our base model and the model ID:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "ce0fcced",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "peft.peft_model.PeftModel"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loaded = peft.PeftModel.from_pretrained(module_copy, model_id)\n",
    "type(loaded)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd4b4eac",
   "metadata": {},
   "source": [
    "Let's check that the two models produce the same output:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "f2cf6ac4",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_peft = peft_model(X.to(device))\n",
    "y_loaded = loaded(X.to(device))\n",
    "torch.allclose(y_peft, y_loaded)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eeeb653f",
   "metadata": {},
   "source": [
    "### Clean up"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61c60355",
   "metadata": {},
   "source": [
    "Finally, as a clean up step, you may want to delete the repo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "b747038f",
   "metadata": {},
   "outputs": [],
   "source": [
    "from huggingface_hub import delete_repo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "7e5ab237",
   "metadata": {},
   "outputs": [],
   "source": [
    "delete_repo(model_id)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
